Task: Organise The Management (AST)
Purpose
To establish the way in which the management of the test process, infrastructure, test products and defects are organised.
Relationships
Main Description

Method of operation

The method of operation covers the following steps:

  1. Defining test process management
  2. Defining infrastructure management
  3. Defining test product management
  4. Defining defects management.

At test plan level, norms and standards can be set up for this, supported by procedures, templates and tools (test management tools, plans and progress monitoring tools). Sometimes at the overall level facilities are arranged to be used.

Products

A description of the various management processes, established in the test plan.

Tools

  • Defect management tool
  • Testware management tool
  • Planning and progress monitoring tool
  • Workflow tool
Steps
1. Defining test process management

Test process management is aimed at administering the test process in terms of progress and quality, and providing insight into the quality of the test object. To this end, identification, registration, administration, storage and interpretation of the following details has to take place:

  • Progress and the expenditure of budget and time
  • Quality indicators
  • Test statistics

This management is sometimes assigned to a dedicated role: test project administrator (see Roles Not Described As A Position).

This information forms the basis for managing and reporting by test management. Since control over the test process is increasing in importance, management is under pressure regarding the test process. Fast – preferably real-time – insight is required into the actual status quo. In this connection, the term dashboard is used: a simple overview from which all the superfluous information is removed and that provides the most important information at a glance: the quality of the test object (in terms of defects) and the progress of the test process. Planning and progress monitoring tools but also testware management tools can be an excellent support here.

Below is an example:

(n.a. = not applicable)

Progress and expenditure of budget and time

The progress information offers the client and the test management insight into the test process. On the basis of this, the test process can be redirected, if necessary. Where there are negative trends, timely measures can be adopted. The parts to be managed are the activities and/or products, related to hours, resources, timeline and with mutual dependencies.

Quality indicators

The aim of testing is to provide information and advice on the risks and quality of the object to be tested. To be able to provide this information, quality indicators are registered. The best-known and most obvious indicator is the defect. By establishing all kinds of details on a defect, such as e.g. status, severity, cause, quality characteristic and system part, all kinds of qualitative information can be gleaned from the defects at a later stage. Bear in mind the number of open defects relating to a particular part of the system, the number of defects found in a particular period, the number of defects relating to the requirements, etc. For more information on defects, refer to Defects Management. Various other indicators are also possible. For example, the number of retests or the number of breakdowns within the test infrastructure (as an indicator of its reliability). The above-mentioned indicators tell us something about the quality of the test object. Another group of indicators tells us something about the quality of the test process itself. For example (see Report (AST)):

Effectiveness of testing Are the (important) defects being found?
Efficiency of testing Are the defects being found as quickly and cheaply as possible?
Checkability of testing Is the test process progressing transparently and in the agreed way?


Test statistics

The test manager builds statistics based on the above information. Statistics can supply insight into the progress of the test process and quality of the test object, including any trends. And statistics can also apply to the quality of the test process itself.

2. Defining infrastructure management
The test infrastructure is subdivided into three groups of facilities:

  • Test environment
  • Test tools
  • Office setup

The test infrastructure is specified and ordered during the early stages of the test process. After installation, intake and acceptance of it, the infrastructure has to be managed. In practice, the management is usually transferred to a department, such as system management or operations, whether or not the test infrastructure coordinator forms the communication channel between the test process and the managing department.

3. Defining test product management

At test-plan level, norms and standards are set up for the management of the test products, supported by procedures, templates and tools. This promotes the reusability of the products and communication on it. It is advisable to adopt the norms and standards generally applied within the system development process to documentation and configuration management. Test product management is sometimes assigned to a dedicated role: testware administrator (see Roles Not Described As A Position).

The following are the various product groups to be managed:

  • Products such as testware and test-project documents. Generally, higher requirements are set in respect of the management of reusable products like testware, e.g. that versions are retained. See also Define The Test Products (AST).
  • External products, such as the test basis and the test object. Responsibility for the management of this lies outside of the test process. However, the importance of good (version) management is extremely important to the test process. For that reason, requirements are often set from within the test plan in respect of the external management – e.g. that each product should be uniquely identifiable.

A choice has to be made as to which products are to be managed and to what degree. The management can be effectively supported by means of testware management tools.

4. Defining defect management
A defects procedure should be set up to facilitate the handling and managing of defects. Ideally, this procedure is supported by a tool. Since a defects procedure applies to the entire project and not to a separate test level, this procedure can best be defined at master test plan level. This also makes it possible to detect overall trends, over and above test levels. A description of the defects procedure is included in Defects Management. This management is sometimes assigned to a dedicated role: defects administrator (Roles Not Described As A Position).
More Information